Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

rocmPackages_6: fix builds by using python311 #324235

Closed
wants to merge 1 commit into from
Closed

rocmPackages_6: fix builds by using python311 #324235

wants to merge 1 commit into from

Conversation

doronbehar
Copy link
Contributor

@doronbehar doronbehar commented Jul 3, 2024 •
edited by superherointj
Loading

In the attempt to help #322145, see:

https://cache.nixos.org/log/6pb1csgh8d3s23hm4gnn8m9468ziwsis-rocm-llvm-compiler-rt-6.0.2.drv

Description of changes

  • rocmPackages_6.llvm.compiler-rt: 6.0.2 -> 6.1.2
  • rocmPackages_6.llvm.compiler-rt: fix build with python3.12

Things done

Haven't tested this. Will wait for ofborg..

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

CC @vcunat

Add a 👍 reaction to pull requests you find important.

Fixes #325907 #325909

@doronbehar doronbehar force-pushed the pkg/rocm-fix-distutils-python3.12 branch from fb27f30 to 65af24e Compare July 3, 2024 08:15
@doronbehar doronbehar changed the title rocmPackages_6.llvm.compiler-rt: update; fix build rocmPackages_6.llvm.compiler-rt: fix build Jul 3, 2024
@doronbehar
Copy link
Contributor Author

I removed the update commit, as it seems to be too involved according to the ofborg errors.

@doronbehar doronbehar force-pushed the pkg/rocm-fix-distutils-python3.12 branch from 65af24e to 3cff9b9 Compare July 3, 2024 09:38
@doronbehar
Copy link
Contributor Author

@ofborg build rocmPackages_6.llvm.compiler-rt

@doronbehar doronbehar changed the title rocmPackages_6.llvm.compiler-rt: fix build rocmPackages_6: fix builds by using python311 Jul 3, 2024
@doronbehar
Copy link
Contributor Author

OK I don't know how to fix these packages... Quoting last ofborg error:

FAILED: docs/CMakeFiles/docs-llvm-html /build/source/llvm/build/docs/CMakeFiles/docs-llvm-html 
cd /build/source/llvm/build/docs && /nix/store/ll2b6cslly4v30im7qk34f9y21ziiz1i-cmake-3.29.6/bin/cmake -E env /nix/store/sr1q620p6mv77ihx6cikihxi7g6a4xbg-python3.12-sphinx-7.3.7/bin/sphinx-build -b html -d /build/source/llvm/build/docs/_doctrees-llvm-html -q -t builder-html -D version=17 -D release=17.0.0git -tPreRelease /build/source/llvm/docs /build/source/llvm/build/docs/html
Configuration error:
There is a programmable error in your configuration file:
Traceback (most recent call last):
  File "/nix/store/sr1q620p6mv77ihx6cikihxi7g6a4xbg-python3.12-sphinx-7.3.7/lib/python3.12/site-packages/sphinx/config.py", line 509, in eval_config_file
    exec(code, namespace)  # NoQA: S102
    ^^^^^^^^^^^^^^^^^^^^^
  File "/build/source/llvm/docs/conf.py", line 40, in <module>
    import recommonmark
ModuleNotFoundError: No module named 'recommonmark'

@doronbehar doronbehar marked this pull request as draft July 3, 2024 09:51
@doronbehar doronbehar mentioned this pull request Jul 3, 2024
Merged
2 tasks
@ofborg ofborg bot requested a review from kira-bruneau July 3, 2024 10:32
@vcunat
Copy link
Member

vcunat commented Jul 3, 2024

You could get distutils by adding setuptools (not sure if normally suffices or by withPackages). But no idea if there are other incompatibilities as well.

@doronbehar
Copy link
Contributor Author

You could get distutils by adding setuptools (not sure if normally suffices or by withPackages). But no idea if there are other incompatibilities as well.

I tried that but it failed with the same original distutils missing error. I didn't try using withPackages though...

SuperSandro2000
SuperSandro2000 reviewed Jul 3, 2024
View reviewed changes
pkgs/development/rocm-modules/6/default.nix
@@ -4,7 +4,7 @@
, symlinkJoin
, fetchFromGitHub
, cudaPackages
, python3Packages
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would recommend to overwrite python3Packages with python311Packages in this file, instead of every other file.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would recommend to overwrite python3Packages with python311Packages in this file, instead of every other file.

I was thinking about it, but it turned out to be a bit harder then running a sed command, because there is no newScope in this file.. I'm open to suggestions though.

@vcunat vcunat changed the base branch from staging-next to master July 6, 2024 07:06
@Naxdy
Copy link
Contributor

Naxdy commented Jul 9, 2024

Even with a custom scope like this:

nixpkgs.overlays = [
  (self: super:
    let
      python311Scope = self.lib.makeScope self.newScope (scope: {
        python3 = self.python311;
        python3Packages = self.python311Packages;
        sphinx = scope.python3Packages.toPythonApplication scope.python3Packages.sphinx;

        rocmPackages = self.recurseIntoAttrs (scope.callPackage "${nixpkgsLocation}/pkgs/development/rocm-modules/6" { });
      });
    in
    {
      rocmPackages = python311Scope.rocmPackages;
    })
];

I'm also still getting complaints about recommonmark not being found. There seems to be something else going on here...

@superherointj superherointj mentioned this pull request Jul 9, 2024
Merged
13 tasks
@ParetoOptimalDev
Copy link 

elf.lib.makeScope

I was also trying to figure this out and tried searching for examples that could shed some light but nothing became obvious after looking at:

https://sourcegraph.com/search?q=context:global+lang:nix+type:file+makeScope+AND+python&patternType=keyword&sm=0

@corpix
Copy link
Contributor

corpix commented Jul 10, 2024

@doronbehar I've fixed error you mentioned #324235 (comment) with following patch:

diff --git pkgs/development/rocm-modules/6/default.nix pkgs/development/rocm-modules/6/default.nix
index 8badd3990d57..203883bbd8f7 100644
--- pkgs/development/rocm-modules/6/default.nix
+++ pkgs/development/rocm-modules/6/default.nix
@@ -17,7 +17,10 @@ let
   rocmUpdateScript = callPackage ./update.nix { };
 in rec {
   ## ROCm ##
-  llvm = recurseIntoAttrs (callPackage ./llvm/default.nix { inherit rocmUpdateScript rocm-device-libs rocm-runtime rocm-thunk clr; });
+  llvm = recurseIntoAttrs (callPackage ./llvm/default.nix {
+    inherit rocmUpdateScript rocm-device-libs rocm-runtime rocm-thunk clr;
+    inherit (python311Packages) callPackage;
+  });
 
   rocm-core = callPackage ./rocm-core {
     inherit rocmUpdateScript;

@doronbehar
Copy link
Contributor Author

Closing in favor of #325909

@doronbehar doronbehar closed this Jul 10, 2024
@doronbehar doronbehar deleted the pkg/rocm-fix-distutils-python3.12 branch October 30, 2024 06:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@SuperSandro2000 SuperSandro2000 SuperSandro2000 left review comments

@acowley acowley Awaiting requested review from acowley

@Flakebi Flakebi Awaiting requested review from Flakebi

@mschwaig mschwaig Awaiting requested review from mschwaig

@lovesegfault lovesegfault Awaiting requested review from lovesegfault

@Madouura Madouura Awaiting requested review from Madouura

@kira-bruneau kira-bruneau Awaiting requested review from kira-bruneau

Assignees
No one assigned
Labels
10.rebuild-darwin: 1-10 10.rebuild-linux: 101-500
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Build failure: rocmPackages.llvm.compiler-rt-6.0.2
6 participants
@doronbehar @vcunat @Naxdy @ParetoOptimalDev @corpix @SuperSandro2000